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I. INTRODUCTION 
1.1 Context 


Assessing the maturation of both new technologies and 
their interfaces in a system is critical to the success of 
The attendance of 


advanced development projects. 


www.ijaers.com 





Abstract—Integrating a space system is an iterative process aimed at 
assembling the system according to user requirements and demonstrating 
through tests that the system is apt to operate in the target environment. 
The integration effort varies from system to system, and there is no 
standard methodology for its accomplishment. Researchers have proposed 
different approaches and metrics to estimate the effort associated with 
integrating a system. Sauser introduced the concept of an integration 
readiness level metric (IRL) for a system as a maturity assessment of the 
integration effort. This methodology parallels the technology readiness 
level (TRL) introduced by NASA in the 80s to assess the maturity of new 
technologies in a space system. The authors propose a methodology for 
the risk assessment of a system’s internal interfaces that translates and 
quantifies the perception of work teams about the challenges involved in 
the integration of a system. It consists of a structured survey of specialized 
opinion among project technical personnel to capture relevant technical 
and programmatic risks related to integrating system elements. The 
framework is designed to support the processes associated with choosing 
a system architecture. Although proposed for application in the early 
phases of the life cycle of a space system project, there are no restrictions 
on its application to different projects and life cycle stages. Applications 
include ranking systems components and interfaces according to 
readiness level and integration risks, supporting risk management in 
conventional risk management processes and supporting risk-informed 
decision-making processes to select system architectures. 


performance, schedule and budget requirements are central 
to project execution. 


Between phases 0 and B of the life cycle of a space 
project, the project team carries out the following tasks: 
definition of mission requirements, identification of 
architecture concepts, feasibility analysis, identification of 
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activities and resources, evaluation of initial technical and 
programmatic risk, elaboration of technical and functional 
requirements, among others. During these phases, the 
decisions that commit a large part of the project's resources 
occur. 


Early identification of the risks and uncertainties of 
these decisions may avoid critical failures affecting 
performance, schedule, or budget. In the initial phases, 
thus, managers shall have the necessary technical support 
for identifying risks and putting in place strategies for 
minimizing them, mainly the risks that have the potential 
for adverse developments in performance, cost, and 
schedule. 


A study carried out by Reeves et al. [1] with two 
National Aeronautics and Space Administration (NASA) 
projects has shown that among 135 identified risks, most 
are internal to the project. Many of these risks are usually 
related to technical activities, including system integration. 
The study demonstrates the importance of decision-making 
in a project, mainly in the initial phases. 


Risk management methodologies may vary from one 
organization to another. NASA preconizes the practice of 
risk-informed decisions in all instances that affect the 
security, performance, cost, and execution time of a 
mission[2]. 


Among the decisions mentioned above, the choice of 
the system architecture deserves special attention. The 
option for a given architecture presents long-lasting 
repercussions in a project and should be carried out within 
the best available technique and information. The main 
architecture challenges that have the potential to impact 
performance, cost, and time must be identified and 
mapped. Among them, one finds the definition and 
selection of the architecture’s constituent parts and the 
difficulties posed by their integration. 


The integration of a space system is an iterative 
process, with a significant number of concurrent tasks 
carried out by a multidisciplinary team, aimed at 
assembling the system according to user requirements and, 
at each stage, demonstrating through tests that the system 
is apt to operate in the target environment. The integration 
effort may vary from system to system, and there is no 
standard methodology for its accomplishment. 


Researchers have proposed different frameworks and 
metrics to estimate the maturity of system components and 
interfaces. In general, such metrics attempt to quantify the 
integration effort and the maturity of the whole set of 
interfaces. In this line, Sauser et al. [3] introduced the 
concept of an integration readiness level metric (IRL) for a 
system. This methodology parallels the technology 
readiness level (TRL) metric introduced by NASA in the 
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80s[4] to assess the maturity of new technologies in a 
space system. 


Researchers have dedicated significant effort to 
developing methodologies to identify and quantitatively 
define the risks associated with the evolution of the 
maturity of the elements of a system. Often, difficulties 
regarding the technological advancement of system 
components and interfaces and their subsequent infusion 
may result in schedule delays, cost overruns, cancellations, 
or failures[5].The Research and Development Degree of 
Difficulty (R&D3) metric[6, 4]estimates the probability of 
failure of an R&D project. Together with the TRL 
assessment of the project, which in the methodology gets 
related to the impact of the project’s failure, the R&D3 
metric provides the necessary input elements for a 
project’s risk management. The Advanced Degree of 
Difficulty Assessment (AD2) framework [5] extends the 
above approach. It considers the scenario of the project of 
a system. It attempts to provide basic information for the 
project’s Technology Development Plan and improve the 
management of the projects’ cost, schedule, and risk [7]. 


This article explores a methodology for assessing 
integration risks, which shows similarity to the AD2 
framework. It ponders integration maturity indices with 
risk factors, aiming at translating and quantifying the 
perception of work teams about the challenges accruing the 
integration work of a system. 


1.2 Problem 


Risk identification is commonly undertaken during the 
early design phases but often fails to identify all events and 
circumstances that challenge project performance. Risk 
events associated with the maturation of the technologies 
considered during the design of a system and those 
associated with system integration may have significant 
repercussions on the cost and schedule of the 
corresponding project. Therefore, it is imperative to (a) 
identify the maturity level of technologies and interfaces, 
(b) identify the risks associated with the maturation of the 
technologies and interfaces that have a maturity level 
below that defined as a minimum for infusion in the 
project and (c) identify the risks associated with the 
integration of the system and its elements. For instance, 
NASA’s projects for space systems require a minimum 
TRL index of 6 for all technologies and interfaces 
belonging to a system baseline architecture[8, p. 1]. 


In project communications, managers should 
emphasize attributions, interaction, integration, and 
reliability. Otherwise, significant problems are more likely 
to occur, especially those linked to interfaces [9]. 


The problems that this work proposes to address may 
be summarized as follows: 
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a) the mismatch between expectations of project 
managers and work teams about the resources required to 
deal with the integration of system elements; 


b) lack of identification or poor understanding of the 
risks related to schedule and cost that the decision-maker 
is accepting when adopting a specific system architecture 
solution; 


c) lack or insufficiency of communication when 
considering competing alternatives and their associated 
risks. 


1.3 Objective 


In this article, the authors propose an interface risk 
assessment methodology that weighs integration maturity 
indices with risk factors, with the latter conveying the 
perception of work teams about the challenges and risks 
involved in the integration of a system. 


The methodology consists of a structured way of 
capturing a broad spectrum of technical and programmatic 
risks related to integrating system elements. The proposed 
risk assessment methodology will, in principle, be capable 
of addressing the technical and programmatic difficulties 
foreseen in the integration of a system. 


The generated information will aid project managers 
when comparing different architectural solutions since 
most of the architecture problems, potentially impacting 
both cost and time, will, in principle, have been brought to 
light. The information will also assist managers in design 
decision-making and purchase decisions, such as selecting 
sources for acquiring equipment or subsystems. 


Il. LITERATURE REVIEW 


The TRL index measures the maturity of individual 
system technologies at a given time in a project. Its 
computation at different phases of a project enables an 
assessment of the evolution of the work performed 
throughout the project’s phases. In the development and 
production of flight systems, TRL assessments are usually 
carried out until the end of Phase B. Afterwards, data from 
the engineering, qualification, and flight models at 
different system levels typically provide enough 
information for assessing the maturity evolution of 
technologies [10, p. 15]. 


NASA adopted a seven-level Technology Readiness 
Level (TRL) metric to assess the development of a 
particular technology in the 1980s [11].The metric evolved 
into the current nine levels metric in the 1990s. Since then, 
the TRL methodology has been widely used at NASA as a 
systematic metric/measurement system to assess the 
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maturity of a particular technology and enable the ranking 
of different technologies concerning their maturity[8]. 


According to the prescription by NASA [12, p. 3], 
typically, new technology conception takes place from 
TRL 1 to 3, and development and demonstration from 
TRL 4 to 6. After reaching TRL 6, new designs would 
follow the usual engineering development cycle, which 
involves building and testing engineering and qualification 
models. After qualification, the system reaches a readiness 
level for flight with a TRL 8 level assignment. After 
successful launching and in-orbit operation, the system is 
finally assigned a TRL 9 level. 


The measure provided by the TRL methodology may 
have different applications: project 
communication; setting of a target/success criterion; 


internal 


project planning; technology selection; communication or 
establishment of integration arrangements; portfolio 
management; cost estimation; risk indicator; and guide or 
measure for engineering development before Preliminary 
Design Review (PDR)[10]. Fig. Idisplays the process 
currently commended by NASA for TRL assessment at a 
given project instant. Space projects typically involve a 
blend of either existing, evolving, or advanced 
technologies [13]. 


The TRL of a system technology does not measure the 
difficulty of integrating this technology into an operational 
system [13, 14]. TRL is a measure of the maturity of 
individual technology. The maturity assessment of several 
technologies integrated into a system is not given by a 
trivial composition of the individual TRLs. Knowledge of 
individual TRLs does not provide insight into the 
components’ integration maturity or the resulting system's 
overall maturity. Yet complex systems have an appreciable 
chance of failing at integration points [15]. 


The question of computing the maturity of a system 
from the maturity of its component technologies, including 
system-specific characteristics, such as integration 
maturity, has received significant attention. The proposed 
frameworks vary in how the TRL of system individual 
technologies are compounded with measures of system- 
specific characteristics, such as integration and 
manufacturing readiness, to produce a whole-system 
readiness level. NASA preconizes the TRL of a system as 
being determined by the lowest TRL present in the system 
[12, p. 26]. The European Cooperation for Space 
Standardization (ECSS) equally preconizes that “... a TRL 
can only be reached by an element if all of the sub- 
elements are at least at the same level ...”[{16, p. 17]. 
Sauser et al. [8] introduced the concept of System 
Readiness Level (SRL),which gives the maturity of a 
system as a composition of the maturity of the system 
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components with the maturity of the interfaces between the 
components. The latter is given by a matrix, termed 
integration readiness level (IRL) matrix, having as its 
elements the maturity assessment of each interface 
between pairs of technologies. Just as the TRL is used to 
assess the maturity of developing technologies, the IRL is 
used to assess the maturity of integrating these 
technologies. Formally, the SRL is computed as a 
composition of the IRL and TRL metrics [8, 17, 18]. The 
IRL scale has evolved over the last decade, with the most 
recent improvements published by Austin and York [19, 
20]. More specific studies aimed at understanding the SRL 
dependency or sensitivity on a given component 
technology, in terms of this technology’s impact on the 
readiness, cost, and overall performance of the system, are 
provided by Gove et al. [21]. 


There has been a great deal of discussion regarding a 
proper definition and method of computation of the SRL 
index of a system, with the publication of several different 
proposals [22, 23, 24, 25, 26, 27]. In particular, Ross [27] 
has proposed a framework that differentiates between the 
technology readiness level of an isolated technology and 
its maturation status concerning insertion in a given 
system. The readiness level of a given technology is taken 
as dependent on the target system's maturity characteristics 
considered for its insertion, such as its integration 
maturity. In this framework, the TRL of each component 
technology is multiplied by indices, with values between 
zero and one, measuring component promptness for 
integration and manufacturing. The whole-system maturity 
readiness level is then taken as the average of the 
computed maturity levels of the system components. 


The TRL assessment at different project milestones 
gives an overview of technology maturity progression 
towards a planned application. However, the knowledge of 
TRL values at varying instants of the life cycle does not 
provide enough information for assessing technology 
development risks [10, p. 40].Differences in TRL indices 
do not measure either the effort or the risk involved in 
moving from one TRL level to the next in an R&D 
development [4, 10, p. 40]. Hence, other metrics have been 
proposed to assess technology risks and effort for 
progression in a project. 


Mankins [6, 4] has proposed the Research and 
Development Degree of Difficulty (R&D3) metric, which 
gives a qualitative estimate of the probability of failure for 
an R&D project with an assignment scheme containing 
five difficulty levels. The approach proposed by Mankins, 
termed Technology Readiness and Risk Assessment 
(TRRA), combines the R&D3 probability with a 
qualitative estimate of the impact of project failure (F). 
Using the pair (R&D3, F), a diagram like the probability x 
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impact matrix in risk management is constructed. Risk 
classification for technology maturation is then 
implemented, as illustrated in Fig. 2. The impact of project 
failure is taken as proportional to the difference between 
the current and target TLR, multiplied by a factor referred 
to as Technology Need Value (TNV), which gives a 
measure of the expected impact of a failure in making the 
considered technology available for future programs. The 
TRRA framework considers the scenario of a technology 
project inside an R&D program. The framework must be 
adapted in several ways when considering the scenario of 
the project of a system that involves innovative 
technologies, such as developing and manufacturing a 
satellite. There is, for instance, the parallel development of 
different technologies to be integrated into the system, 
changing TNV assessment definitions. 


The Advanced Degree of Difficulty Assessment (AD2) 
methodology by Bilbro [5] expands theR&D3 scheme by 
combining the information from several maturity level 
metrics, such as Integration Readiness Levels, System 
Readiness Levels, Manufacturing Readiness Levels, and 
others. In the AD2 methodology, the number of difficulty 
levels is increased from five to nine, in line with the 
current TRL scale. 


identify systems, sub- Assign TRL to subsysterns 


Assign TRL to all 


systems, and components >|) narinas badon. h > based on lowest TRL of 
per hierarchical product Spotless of maturity components and TRL 


breakdown of the WBS state of integration 


| 


Assign TRL to systems 
based on lowest TRL of 
subsystems and TRL 
state of integration 


identify all components, 

Baseline technology < subsystems, and systems 
maturity assessment that are at lower TRLs 

than required by program 


v 


Perform AD? on all 
components, subsystems, 
and systems that are below 

requisite maturity level 


v 


Technology Development Plan 
Cost Plan 
Schedule Plan 
Risk Assessment 


Fig. 1: Recommended NASA technology assessment 
process [28] 


The AD2 framework considers the scenario of 
developing a system with a broad range of possibilities 
regarding the component technologies, from new 
developments to the partial or total reuse of heritage 
technologies. 
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Fig. 2: Mankins [4]. A typical result of the Technology 
Readiness and Risk Assessment (TRRA) methodology. 
Each technology is characterized by a point in the 
diagram. Regions with different colors provide the risk 
classification for each technology 


The system architecture defines the system's physical 
building blocks in terms of functions and interfaces. The 
project team chooses the system architecture by the end of 
Phase A and the beginning of Phase B. Usually, there are 
two or more paradigmatic proposals. For each proposal, 
several possible variants spring up from the possibilities 
opened by, for instance, alternative technologies, make or 
buy tradeoffs, reuse of equipment or subsystems, 
alternative subsystem or equipment providers, and so 
forth. Careful system decomposition and decision-making 
processes are normal activities at this stage, which 
precedes the system requirements review (SRR) when 
subsystems’ preliminary design receives full attention. 
After the SRR, decomposition and decision-making 
processes are still regular activities, but now at the 
subsystem level. 


To support decision-making processes and control and 
monitor the project’s progress during these phases, 
engineering managers usually make use of different 
metrics, which may be used as input both for RIDM 
processes and the evaluation of the preliminary and critical 
design efforts. The metric of Technology Readiness Level 
has been successfully applied within this context in 
different organizations, mainly to assess the maturity of 
technologies envisaged for incorporation into a system or 
subsystem[29, p. 281]. The TRL index gives the readiness 
or maturity of a single equipment/technology at a given 
time instant. The index results from an assignment by 
qualified personnel using a standard TRL scale. 


RIDM processes are carried out whenever decisions 
involve high stakes, complexity, uncertainty, multiple 
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attributes, or diversity of stakeholders [30]. They are 
conducted with the intervenience of subject experts, 
technical authorities, stakeholders, and the decision-maker 
[30]. 


Risk management methodology may vary from one 
organization to another. At NASA, risk management is an 
integral part of the Systems Engineering process[28]. It 
emphasizes the use of risk analysis in a broad sense, 
promoting the practice of risk-informed decisions in all 
instances that affect the security, performance, cost, and 
execution time of the mission. 


The NASA Risk Management Process (RM) integrates 
two complementary processes in a single framework: Risk- 
Informed Decision Making and Continuous Risk 
Management (CRM). The RIDM process addresses the 
selection of a given alternative among a set of options 
based on the qualitative and quantitative (probability) 
assessment of each choice by a qualified panel to ensure 
the selection process achieves performance objectives. The 
RIDM process is applicable throughout the project life 
cycle whenever critical tradeoff decisions, such as 
architecture and design decisions, make-buy decisions, 
source selection in significant procurements, and budget 
reallocation, are demanded [30, p. 13]. The CRM process 
deals with the selected alternatives’ risk management, 
planning actions to mitigate and control the associated 
risks, and avoiding performance shortfalls[30, p. 14]. 


Table. 1 gives a brief description of different studies 
and corresponding references concerning the concepts of 
TRL, SRL, and risk associated with technology maturation 
in a project/program. 


Table. 1: Assessment techniques 








Study Description 

TRL This quantitative model does not 

Schedule communicate the maturity of technology at 
: a certain point in time but instead leverages 

oe the TRLs metric to identify the appropriate 

Curve schedule margins associated with each 


TRL level to mitigate schedule slips [31]. 








SRL Max The 
mathematical model aiming to maximize 
the SRL under constraint resources. The 
SRL MAX’s objective is to achieve the 
highest possible SRL based on the 
availability of resources such as cost and 
schedule [32]. 


SRL Max is a quantitative 
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Advanced Leveraging the concept of RD3, the AD2 
Degree augments TRLs by assessing the difficulty 
of Difficulty of advancing technology from its current 
(AD2) level to the desired level on a 9-tier scale 

[33]. 

Research The RD3 is a 5-level scale intended to 
And supplement the TRL by conveying the 
Developmen degree of difficulty in proceeding from the 
t Degree of current TRL state to the desired level, with 
Difficulty(R five being very difficult and one being the 
D3) least difficult to mature the technology [6]. 
Technology | “TRRA is a quantitative risk model that 
Readiness incorporates TRLs, the degree of difficulty 
and Risk (RD3) of moving a technology from one 
Assessment( TRL to another, and Technology Need 
TRRA) Value (TNV). The TRRA expands the 


concept of the risk matrix by integrating” 
the "probability of failure" on the y-axis 
and the "consequence of failure" on the x- 
axis[34]. 











mM. METHODOLOGY 
3.1 Description of the proposed framework 


We consider the situation of the project of a space 
system that incorporates new technologies during the end 
of Phase A and the beginning of Phase B, when decisions 
with long-lasting impact are to be taken, such as the 
definition of the system's architecture. The availability of 
TRL values at different instants of the project's life cycle 
does not provide information on the risks associated with 
technology maturation. It should be supplemented with 
risk assessments, making use, for instance, of such 
schemes as the AD2 framework. 


The expression "architecture solution" in this article is 
defined as the set of specific elements (products Part 
Numbers) that make up the physical architecture. To 
compare different architecture solutions, we consider that 
an assessment of integration risks shall complement the 
TRL data of system components and the IRL data of 
interfaces; additionally, an evaluation of the technology 
maturation risks pointed out above should be available. 
Typically, these data are fed into RIDM processes that 
substantiate decision-making activities. 


Analyses of the work of integrating each interface, 
considering all candidate equipment, shall focus, for each 
alternative, on the following points: the type of the 
integration, i.e., electrical, mechanical, thermal, signal, 
etc.; the risk of delays in the program; the risk of an 
increase of the initially estimated cost for the realization of 
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the system; the identification of difficulties and challenges 
associated with each alternative. 


This routine must be applied to the complete system, 
considering all supply options. In this way, a system 
integration risk assessment is produced for each 
architecture solution, giving visibility to the system 
compositions with the most significant potential for 
adverse developments regarding the performance of the 
work of integration, schedule, and cost. 


The information gathered will typically make part of 
the input data package for the tradeoff study to select the 
most balanced system architecture, possibly within the 
scope of a RIDM process. Once a choice has been made 
among the options, the identified risks must be addressed 
and mitigated, typically in a project's CRM risk 
management process. 


The integration risk assessment shall be carried out for 
all interfaces of each architecture configuration by 
specialists involved in the project and performed 
separately. 


The assessment is carried out for a specially selected 
set of integration risks, chosen according to the experience 
of one of the authors (HES)in space projects and through 
interviews with systems engineering teams, specialist 
engineers, project managers, and Assembly, Integration, 
and Testing (AIT) teams of the Brazilian space program. 
They constitute a set of risks that cover most of the causes 
for cost and schedule increases in space systems 
development. 


Table. 2 lists the selected risks in the format used in the 
assessment. The questions were designed in the form of an 
ordinal unipolar survey [35]. Each general risk event is 
unfolded into three excluding possibilities, with the 
leftmost possibility being taken as the zero point option. 
The general and unfolded risk options were carefully 
chosen to explore project integration risks localized in the 
relevant severity region highlighted in Fig. 3. The three 
possibilities are associated with different impacts of the 
corresponding general risk. Selection of the first option, 
referred to as Negligible (N), indicates that the assessor 
considers that the event associated with the stated general 
risk has a low likelihood or no chance of occurring. 
Selection of one of the other two possibilities, referred to 
as Low (L) and Moderate to High (M/H), indicates that the 
evaluator considers the risk relevant, with severity in the 
range from low to medium (option 2) or from high to 
extremal (option 3). The assignment of option 1 to a given 
risk indicates that the evaluator assesses its likelihood at 
less than 30%. Table. 3 shows the assumed likelihood 
levels[7]. 
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Table. 2: Classification criteria of the expected difficulty in integration 


























s Unfolded Risk Options 
Risk Statement a i 
N - Negligible L - Low M/H - Moderate to high 
Given that the personnel belonging to the The technological The technological The technological 
following teams: a) system, (b) solutions to be employed integration solutions are | integration solutions are 
_, | manufacturing, and (c) integration might in the interface integration | not thoroughly not thoroughly 
> | not entirely dominate the technological are thoroughly dominated | dominated by the dominated by the 
& | solutions to be employed in the interface by the personnel personnel of one of the personnel of at least two 
> | integration, there might be unanticipated belonging to the following teams: a) of the following teams: a) 
2 | technical difficulties, adversely impacting following teams: a) system, (b) system, (b) 
| the integration work, therefore leading to system, (b) manufacturing, and (c) manufacturing, and (c) 
interface nonconformities and schedule and | manufacturing, and (c) integration team. integration team. 
cost overruns. integration team. 
Given the limited state of knowledge about | There are no envisaged There are anticipated There are anticipated 
the use of ground support equipment and challenges regarding entry-level challenges moderate to high-level 
tooling in the interface integration effort, ground support equipment | regarding ground challenges regarding 
X there is the possibility of unanticipated and tooling in the support equipment and ground support 
2 technical difficulties, adversely impacting integration effort. tools in the integration equipment and tools in 
the interface integration work, therefore effort. the integration effort. 
leading to interface nonconformities and 
schedule and cost overruns. 
: : : ; Any necessary physical Any necessary physical | Any necessary physical 
Given that the integration effort might y ne ry pays y ne ry pays y ne y pays 
(Sa : : f : (electrical, mechanical, (electrical, mechanical, (electrical, mechanical, 
>, | require physical and/or logical adaptations i ; ; 
Z 5 : etc.) or logical adaptations | etc.) or logical etc.) or logical 
© | and that the teams involved might not ; . : er ; RAE 
z : : involve technological adaptations will likely adaptations will highly 
S | entirely dominate the technology for the , ; : ; è à 
> : : solutions available and involve technological likely involve 
necessary adaptions, there might be ` : : i : 
B ie : pepe ays thoroughly mastered by solutions not entirely technological solutions 
a | unanticipated technical difficulties and f : : f . 
5 Me ae : ; the teams involved in the | available or thoroughly | not either available or 
-5 | additional training, adversely impacting the : : 
so |. : : : effort to integrate the mastered by the teams dominated by the teams 
2 | interface integration work, therefore leading : ne ; $ : : i 
z ; 3 studied pair. involved in the effort to | involved in the effort to 
© | to interface nonconformities and schedule ; ; ; ; , 
< integrate the studied integrate the studied pair. 
and cost overruns. : 
pair. 
Given that the workload for the The workload foreseen by | The workload foreseen The workload foreseen 
st | mechanical/electrical/logical integration of | the teams involved in the | by the teams involved by the teams involved in 
2 the interface elements might not be mechanical/electrical/logi | in the the 
5 | accurately predicted, there might be c integration of the mechanical/electrical/lo | mechanical/electrical/logi 
S unanticipated delays, adversely impacting interface elements is gic integration of the c integration of the 
=~ | the allocation of personnel to the considered very low. interface elements is interface elements is 
& | integration work, therefore leading to considered low. considered moderate or 
schedule overruns. high. 
; Pee seu From the perspective of From the point of view From the point of view of 
Given the possibility of limited knowledge P Sp P ; P ; 
i the evaluator’s team, the of the evaluator's team, the evaluator's team, the 
by the personnel from the evaluator's team : 3 . K 
Si SEA scope of the integration the scope of the scope of the integration 
w | about training necessities in the face of the : : : f p 
op : ; : work is well known, and integration work is work is not thoroughly 
= | scope of the integration work, there might ee : i at 
j= hee a on it is highly unlikely that reasonably known, and known, and additional 
-< | be unanticipated training necessities, ne i Be D 
S : à à additional training of the additional, low- training of moderate to 
& | adversely impacting the allocation of : : : a . : i 
. $ personnel will be complexity training for high complexity for the 
personnel to the integration work, therefore : : : À 
; necessary. the teams involved will | teams involved will be 
leading to schedule and cost overruns. i : 
be required. required. 
The history of non-conformities and The history of The history of The history of 
© | recurrences associated with the nonconformities and nonconformities and nonconformities and 
2 | technological solution for the integration of | recurrences does not recurrence suggests the | recurrence strongly 
S the studied pair might indicate the suggest difficulties in the | possibility of suggests the possibility 
z possibility of difficulties in the interface integration work difficulties in the of difficulties in the 
| integration work that will lead to time and associated with the integration work integration work 
3 | cost overruns depending on its level of interface. associated with the associated with the 
5 recürfence interface. interface. 
S ; 
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Interface Requirements 7 


Given the limited knowledge about the 
completeness of the technical requirements 
set, there is the possibility of the need for 
additional physical and logical 
requirements, negatively impacting the 
integration work, leading to time and cost 
overruns. 


The requirement analyses 
indicate that it is highly 
unlikely that additional 
physical (mechanical, 
electrical, etc.) or logical 
requirements will be 
needed during the 
integration work. 


The requirements 
analyses indicate a low 
probability that 
additional physical 
(mechanical, electrical, 
etc.) or logical 
requirements will be 
needed during the 
integration work. 


The requirements 
analysis indicates a 
moderate to high 
probability that 
additional physical 
(mechanical, electrical, 
etc.) or logical 
requirements will be 
needed during the 
integration work. 





Verification Approach 
8 


Given the current state of knowledge about 
the scope of the necessary verification 
work, there might be unanticipated 
additional analyses and tests associated 
with the verification effort, adversely 
impacting the integration work, therefore 
leading to schedule and cost overruns. 


Considering that the 
scope of the verification 
work is well-known, it is 
highly unlikely that 
additional analyses and 
tests associated with the 
verification effort will be 
necessary. 


Considering that the 
scope of the verification 
work is reasonably 
known, there is a low 
probability that 
additional analyses and 
tests associated with the 
verification effort will 
be necessary. 


Considering that the 
scope of the verification 
work is poorly known, 
there is a moderate to 
high probability that 
additional analyses and 
tests associated with the 
verification effort will be 
necessary. 





Complexity 9 





Given the current state of knowledge about 
the complexity of the adopted integration 
technology, there might be an unanticipated 
increase in the workload and training 
necessities, adversely impacting the 
allocation of personnel to the integration 
work, therefore leading to technical 
nonconformities and schedule cost 
overruns. 


Considering that the 
adopted integration 
technology shows very 
low complexity, it is 
highly unlikely that there 
will be an unanticipated 
increase in the workload 
and training necessities. 





Considering that the 
adopted integration 
technology shows low 
complexity, it is likely 
that there will be an 
unanticipated increase 
in the workload and 
training necessities. 


Considering that the 
adopted integration 
technology shows 
moderate to high 
complexity, it is highly 
likely that there will be 
an unanticipated increase 
in the workload and 
training necessities. 











For example, Table. 4 shows the unfolding of the first 
general risk listed in the first column in Table. 2 into three 
event possibilities with different impacts. The leftmost 
option corresponds to the assessment that the risk has a 
low likelihood or is not applicable. The other options 
correspond, by design, to the judgment that the risk is 
relevant, from likely to near certainty chance and severity 


in the range highlighted in Fig. 3. 
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2 - Low 
likelihood 


3 - Likely 









3 4 5 


Moderat| Signific Severe 




















4 - Highly 4 8 
likely 
5 -Near 5 
Certainty 
Minima 
Risk I 
Rating 1-2 | 3-9 








Extre 
High | mal 
10-15 |16-20/ 25 





















Fig. 3: Risk classification used in the proposed framework. 


The highlighted region of the diagram shows the severity 


(probability x impact) region of the considered risks in the 


proposed framework 
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Table. 3: Likelihood levels 


Likelihood Probability of Occurrence 
Norike 


Like 
Highly likely 





An example of a typical evaluation form is given in 
Table. 5.The line Frequency gives the proportion of each 
option N, L, and M/H in the form. The line Weighting 
factors list the weight values for each option; they shall be 
conveniently defined for the computation of a weighted 
average, which will be taken as a measure of the riskiness 
of the integration work associated with the concerned 
interface, as will be discussed ahead. 


From the filled forms, important information, which 
expresses the specialized opinion of the project team 
responsible for most of the integration work’s scope, may 
be extracted from simple statistics. For each selected risk 
m and interface g, by computing overall numbers from all 
forms /, one obtains the fraction of options 0 as given by: 


1 oN 
Yina=y_ 1 mals (D) 


o e{N,L,M/H}, 

m e{1,2,...,9}, 

q= interface identificator, 
where Yq is equal to 1 if the evaluator has chosen option 
o, in the form /, for the risk m; otherwise Yinglis equal to 0; 


Ng is the number of filled forms for the interface q. 


Table. 4: Possible impacts which may emerge from the 
first risk listed in Table. 2 
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interface 
integration are 
thoroughly 
dominated by all 
the following 
teams: a) system, 


(b) 


interface 

integration are 
not thoroughly 
dominated by 
one of the 
following teams: 
a) system, (b) 


interface 

integration are 
not thoroughly 
dominated by 
two or more of 
the following 
teams: a) system, 





manufacturing, manufacturing, (b) 

and (c) | and (c) | manufacturing, 

integration. integration. and (c) 
integration. 














RISK STATEMENT: Given that 
belonging to the following teams: a) system, (b) 
and (c) 


dominate the technological solutions to be employed in 


the personnel 


manufacturing, integration may not fully 
the interface integration, there is the possibility of 
unanticipated technical difficulties, adversely impacting 
the integration work, therefore leading to interface 
nonconformities and schedule and cost overruns. 




















Impact 
2-3 4-5 
Negligible or not ; ee 
applicable (Minor (Significant to 

toModerate) Severe) 
The The The 
technological technological technological 
solutions to be | solutions to be | solutions to be 
employed in the | employed in the | employed in the 
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Table. 5: Example of an integration risk evaluation form. 
The numbered lines are in correspondence with lines in 
Table. 2. The meaning of the acronyms is as follows: N for 
Negligible, L for Low, and M/H for Moderate to High 





Case study identification: 


Evaluator identification: 


Identification of the interface/integration: 


2 - GSE 


3 - Adaptation & Mastery 
4 - Workload 


N L 
5 - Training 
6 - Nonconformance 








Weighting factors 
Criterion's weight 3W, /9 | 40/9 


The expression: 











M 
Img = Wy¥Nq + Wig + Wynne” (2) 


gives an index, with values between Wy and Ww, 
expressing the severity of each considered risk. 
ForlmqnearWy, the risk is considered negligible or non- 
existent, while for I,gnearWy /y, the risk is considered 
severe. By using these indices, it is possible to rank, for 
each interface q, the risks m according to their relevance. 
In this way, from the assessments carried out by each 
specialist, one obtains the set of risks that the specialized 
project team considers most critical in the project’s system 
integration effort. The collection of integration risks 
ranked according to the index Imqą provides an easy and 
convenient way of communicating integration risks across 
the whole project hierarchy. 


It would also be desirable to consolidate the 
information through a representative index for each 
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configuration. This possibility would enable a ranking of 
the solutions and facilitate communication in selecting a 
system architecture. Some of the options are discussed 
below. 


Typically, the project team will have at its disposal, for 
each possible architecture, the information condensed in 


Table. 6. 











Table. 6: Available information for each system 
configuration 
Equipment | Interfaces Maturation Integration 
Risks Risks 
TRL for IRL for Risks related | Most 
each piece | each to the relevant 
of interface, maturation of | integration 
equipment. | considering | the risks, for 
the technologies | each 
different of interface, 
integration | components considering 
types (E, and the different 
M, T, interfaces; integration 
D/C). usually types (E, M, 
identified and | T, D/C). 
characterized 
by an AD2 
analysis. 

















As reviewed in Section II, the concept of a system 
readiness level, SRL, formed from the TRL and IRL 
values of system components and interfaces, is usually 
taken as a measure of the readiness level of the whole 
system. There are different proposed methods for the 
computation of the SRL index. This measure should be 
supplemented by a risk analysis, which is usually 
undertaken under the AD2 methodology preconized by 
NASA. The AD2 method has been designed to identify 
and characterize the risks associated with upgrading a 
system, subsystem, or component from a given TRL to a 
higher one. The AD2 methodology focuses on the 
following risk areas: design and analysis, manufacturing, 
software development, testing, and operations. The 
framework proposed in this work may be interpreted as 
proposing that the AD2 methodology be complemented by 
a risk analysis dedicated to identifying and characterizing 
the risks associated with the integration of a given system 
configuration, providing, in principle, an additional set of 
risks that complements the set of risks provided by the 
AD2 methodology. 


A measure of the integration risk for each architecture 
may be constructed as follows. For each interface and 
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integration type, a Technical Difficulty Factor (TDF) is 
computed from the fractions of N, L, and M/H assessments 
in the complete set of responses as: 


freqg(0) = D OL 2 G 


freq,(o) = (5) >, ie (4) 


TDF, = freq (N) * Wy + freq, (L) * W, + 
freqg(M/H) * Waujn > 


whereY,,,has been defined in (1), Nq is the number of 


(5) 


filled forms, freq, (0)gives the fraction of options o for the 
interface q considering all forms and risks, and q 
designates the different interfaces, including the 
multiplicities introduced by the different integration types. 
The index TDA, with values between Wy and Wmm, 
expresses the severity of each considered risk on the 
integration of interface q: the larger the value ofTDF}, the 
larger the chance of difficulties in the integration of 
interface q, according to the project team. It gives a 
measure of the risk affecting the integration work 
associated with interface q. The set of interfaces ranked 
according to the index TDF, providesan ordered list of the 
interfaces that the specialized project team considers most 
critical in the project’s system integration effort. Similarly, 
as the index I,,, facilitates integration risk communication, 
the index TDF, provides a straightforward way of 
communicating to the project hierarchy which interfaces 
are considered critical in the integration effort. 


Averaging, now, the index TDF, over all interfaces: 


TDF = GÈ TPR, (6) 


where Q is the number of interfaces, including the 
multiplicities introduced by the different integration types, 
defines an index with values between Wy and Wmwu that 
gives a measure of the overall integration risk of the 
configuration for which it has been computed. Hence, 
ordering the studied architectures according to the 
corresponding values of TDF provides a ranked list of the 
architectures, from riskiest to least risky, according to the 
assessment of the specialized project team. Hence, the 
TDF index offers a straightforward way of communicating 
to the project hierarchy which architecture is considered 
most risky regarding the integration effort. 


The indices Ima, TDF,and TDF provide information 
regarding the risks associated with the integration effort of 
each system architecture. In contrast, the indices TRL and 
IRL provide information regarding the maturity of 
equipment and interfaces, respectively. Next, a proposal 
for the composition of these metrics is discussed. 
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The interface attributes expressed by/RL, and TDF, 
provide a measure of the maturity and the integration risk 
associated with interface q, respectively. While the IRL, 
index may be used to rank interfaces according to their 
maturity, theTDF,index may be used to rank interfaces 
according to their integration risk. One question that 
naturally arises at this point is how to rank the system 
interfaces by composing the two attributes/RL, andTDF,. 
The problem of ranking a variable with multiple attributes 
has received much attention [36, 37, 38]. In most 
applications, a weighted multiplicative scoring is 
considered more reliable than a weighted additive scoring 
[38].Along this line, a possible index for ranking 
interfaces, based on the attributes of maturity and 
integration risk, may be constructed from a multiplicative 
scoring function involving the indices of the corresponding 
attributes. 


An appropriate function, with equal weight exponents 
for both attributes, may be expressed as: 


RWIRL, = IRL, x (Wy + Wujn — TDF), (7) 
IRL} E {1,2,...,9}, 
where q designates the considered interface. 


The scale for TDF has been reversed, aligning both 
attributes so that the value of the proposed index increases 
with increasing maturity and decreasing integration risk. 


The RWIRL, index shall be used exclusively for 
ranking purposes. It may be considered as giving for 
interface q, at a given instant, a composite measure of both 
integration maturity and severity of perceived risks for its 
integration. 


Using the RWIRL, indices, it is possible to rank the 
interfaces according to their relevance in terms of maturity 
and integration risks. The set of interfaces ranked 
according to the index RWIRL, provides a convenient way 
of communicating the most critical interfaces, regarding 
maturity and integration risks, to the project hierarchy. 


Averaging, now, the indexRWIRL, over all interfaces: 


1 
RWIRL_int = (3) RWIRL, , (8) 


q=1 
introduces an index that gives a measure of the overall 
integration maturity and integration risks associated with 
the considered system architecture. The risk-weighted 
interface readiness level index,RWIRL_int,is a system- 
wide index that combines overall interface maturity with 
overall interface integration risk. It may have relevance in 
a decision-making process dedicated to selecting a system 
architecture. 
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Next, along the same lines, we discuss the definition of 
a system-wide index that incorporates the maturity of the 
components that define the system architecture. 


Following the previous discussion, a weighted 
multiplicative scoring is defined for each equipment based 
on the attributes of (a) maturity of the equipment, (b) 
maturity of the equipment’s interfaces, and (c) the 
interfaces’ integration risk. Since a piece of equipment 
may have more than one interface, a possible scoring 
function may consider an average for the interfaces, as 


given by: 
Tn 1 
RWSRL; =TRL ; x ~). IRL;j x (Wy + Wun 
Qi fa (9) 
= TDF;;), 
= TRL; x RWIRL;, (10) 


TRL; € {1,2,...,9}, 
IRLi; € {1,2,.., 9}, 


where: 





RWIRL,; = D RWIRL;j, (11) 
tji 

the pair (ij)labels interfaces in the system, identified 
through the system elements between which the interface 
exists, Q; is the number of interfaces of element i IRL; jis 
the maturity of the interface between elements i and j, 
TDF,; is the technical difficulty factor for the interface 
between elements i and j, and TRL;is the technical 
readiness level of element i. A note on the convention is in 
place. Interfaces will be identified either through a simple 
ordinal index q or by a pair of indices (ij) indicating the 
system elements between which the interface exists. Thus, 
if q labels the interface between system elements i and j, 
bothRWIRL, and RWIRL;; designate the same index 
corresponding to the interface between elements i and j. 
The same holds for the indices IRL, and TDA}. In (7), a 
prescription like the one followed by Ross [27]for the 
association of interface maturity with a system element has 
been observed: for a given system element i, the average 
of the indices RWIRL;j, 


element i, referred to as RWIRL;, has been taken as an 


associated with the interfaces of 


“average” interface attribute associated with the element i. 
The index RWIRL; may be used to rank equipment 
according to its interface maturity and the perceived 
difficulty of its integration into the system. 

Like the previous indices, the index RWSRL;shall be 
used exclusively for ranking purposes. The index applies 
to system elements and may be considered as giving, at a 
given instant, a composite measure of (a) equipment 
maturity,(b) integration maturity of the equipment’s 
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interfaces, and (c) severity of perceived integration risks 
associated with the equipment’s interfaces. Hence, through 
theRWSRLyindices, it is possible to rank system 
architecture elements according to their relevance in terms 
of the aforementioned three elements’ attributes. 


An index that compounds, at a system level, the 
attributes of (a) maturity of the system’s equipment, (b) 
maturity of interfaces, and (c) risk associated with the 
integration of interfaces may then be defined by: 


1 ere 
RWSRL = (-) > RWSRL,, (12) 
i=1 


where n is the number of system elements. The RWSRL 
index is defined for each system architecture. It may be 
relevant in a decision-making process dedicated to 
selecting a system architecture when it may be employed 
to rank different architectures. 


Similarly, an index that compounds interface 
maturity and equipment integration risk for a given system 
configuration may be defined as: 


n 
RWIRL,; . (13) 


RWIRL = £) 
n i=1 

Both the indices RWIRL_intand RWIRL compound the 
attributes of interface maturity and interface risk of 
integration at a system level. The difference between them 
lies in the fact that while RWIRL_intis computed over all 
interfaces indistinctly, the index RWIRL iscomputed 
considering the partitioning of the interfaces among the 
system components, as seen from (11). The latter index, 
thus, depends on the configuration of the system and is, in 
principle, a more appropriate measure for ranking 
configurations according to interface maturity and 
perceived risk of integration of the configuration. 


3.2 Step-by-step procedure for application of the 
proposed framework 


Fig. 5 gives the methodology's step-by-step process to 
obtain the RWSRL of each architectural solution and other 
indices. The following sections detail each step. 


3.2.1 STEP 1 


The technical team shall identify the equipment pieces 
in the system with more than one candidate supplier and 
all possible architectural solutions, considering the 
possible combinations of candidate equipment. 


3.2.2 STEP 2 


All interfaces between system elements are identified, 
possibly using an N-squared diagram (N°). The technical 
team then classifies each interface according to the 
integration characteristics depicted in Table. 7. 
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Table. 7: A proposed classification scheme for interfaces 





TYPE OF SUBTYPES DESCRIPTION 
INTEGRATION 





Comprises the mechanical 
joints and fastenings 
between the pairs of 
elements; geometries, 
mass property and 
stiffness matching, etc. 


Mechanical (M) 





Comprises the supply of 
Power (P) [Power (voltage and 
current) and grounding 
between the pairs of 
elements. 


Electrical (E) 
Ground (G) 





Comprises the heat 
exchanges (conduction 
and radiation) and thermal 
insulation between the 


Thermal (T) 


pairs of elements. 








Interface involving the 


Data and Data (D) exchange of digital data 
ata an 


and analog signals, such 
Command (D/C) |Command (C) site 


as telemetry and 
telecommands. 











The classification given in the table provides a 
sufficiently broad characterization that encompasses usual 
integration challenges. Depending on the system, it may be 
convenient to distinguish between subtypes of integrations, 
as shown in Table. 7.Building more than one N2 diagram 
may also be necessary when there is more than one way to 
provide the same function through a different arrangement 
of candidate equipment. 


Fig. 4 gives an example of a system with four elements 
represented in an N2 diagram as a function of the 
integration types between each pair of elements. The 
acronyms are defined in Table. 7. 


Equipment A MTG 
Equipment B 


Fig. 4: Example of mapping integrations types 


Legend: M- Mechanical; T-Thermal; P- Power; G- 
Electrical Ground; D- Data; C-Command. 
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STEP 1 


m> 





STEP 2 





—> 


STEP 3 


L» 


STEP 4 





identification of candidate 
equipment and possible 
architectural solutions. 








Identification of interfaces 

between system elements 

according to the integration 
characteristics. 




















Identification of adaptations 
and modification needs to 
integrate each candidate 

equipment into the system. 





Assigning the technological 
readiness levels (TRL) of all 
system elements. 




















STEP 5 


m 


STEP 6 


> 


STEP 7 


— 


STEP 8 





Assignment of integration 
readiness levels (IRL) 
between pairs of elements 
for each possible 


Construction of detailed interface 


diagrams for architectural 
solutions. 


Assessment of integration 
risks associated with each 
interface. 





Assigning a Technical 
Difficulty Factor (TDF) for 
each identified integration 


technological solution taking 
into account each type of 





and calculating the RWSRL. 





























Fig. 5: Roadmap for using the RWSRL methodology 


integration. 
3.2.3 STEP3 
In this step, considering the requirements and 


preliminary design specifications, the technical team 
identifies adaptation and modification needs for integrating 
each candidate equipment into the system. Repercussions 
over other equipment and subsystems shall also be 
The need for further 
analysis addressing testing of 
qualification status should also be contemplated. Usually, 
such studies are carried out by specialists in system 
engineering and assembly, integration, and testing (AIT) 
from the organization responsible for developing the 
system. 


3.2.4 


considered in this assessment. 


and modification 


STEP 4 


From the information gathered in Steps 2 and 3, the 
technical team shall assign a technology readiness level 
(TRL) index to the system components. According to the 
TRL philosophy, the TRL assessment gives the readiness 
level of each system element at a given moment of the 
project life cycle and for the environment prevailing at that 
moment [16, p. 15]. Hence, when the conditions holding at 
the time of a TRL assessment change, as in the case of 
equipment reuse in a different system, with eventual 
alterations in either the design, development process, 
targeted environment, operations[16, p. 16], a 
reassessment of the TRL index will be necessary. 


or 


The TRL assignment may be carried out according to 
the classification given in Table. 8, which gives the ISO 
and ECSS standards for TRL classification. 
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Table. 8: ECSS-E-HB-11A [19] TRL scale 








ISO 16290 standard 





1 Basic principles observed and reported. 





Technology concept and application formulated 








2 
3 Proof-of-concept 
4 


Component and breadboard functional verification 
in a laboratory environment 





5 Component and breadboard critical function 
verification in a relevant environment 





6 Model demonstrating the critical functions of the 


element in a relevant environment 





7 Model demonstrating the element performance 
for the operational environment 





8 Actual system completed and accepted for flight 


(“flight qualified") 





9 Actual system "flight-proven" through successful 
mission operations 











3.2.5 STEP 5 


The technical team shall assign an interface readiness 
level (IRL) index to each identified interface, considering 
all possible architectures. The integration readiness levels 
(IRL) assignment for each interface type (M, E, T, and 
D/C), between each technology pair, shall consider the 
analyses carried out in Step 3. The assignment of an IRL 
index for each interface must be followed by evidence. 
Just as changes influence the TRL index, the IRL index is 
also susceptible to changes in the environment and must be 
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reassessed in the event of modifications. Hence, the same 
pair of technologies may display different IRLs, either as a 
function of the type of integration (M, E, T, or D/C) or due 
to varying choices of candidate equipment. In summary, 
all integrations mapped in STEP 2 and all architectural 
solutions must undergo an evaluation regarding their 
maturity index. An example of decision criteria for the IRL 
assignment is given in Table. 9. 


3.2.6 STEP 6 


The technical team shall construct detailed interface 
diagrams for each system architecture, considering the 
different interface classifications given in Table. 7 (M, E, 
T, and D/C). The charts will be employed in identifying 
and analyzing the technical difficulties associated with 
each interface in the following steps. Fig. 6 displays an 
example of interface diagrams for a four-element system. 


3.2.7 STEP7 


The technical team shall assess the integration risks 
associated with each interface using the assignment criteria 
given in Table. 2. The risk assessment must be carried out 
by subject matter experts and performed separately for 
each specified architecture. The assignment procedure 
makes use of specialized opinion. The evaluator goes 
through the nine criteria given in Table. 2 and identifies 
the best assessment for each interface. This procedure 
must be performed for all mapped integrations in the 
system. Table. 5shows a typical assessment form. 


Mechanical interface(M) 


Thermal Interface (T) 


l 





Data/Command 
Electrical Interface (E) Interface (DC) 
— a, / N fo N 


A enre B |) / | A <—IAL Do B \ 





Fig. 6 Example of decomposition of integrations 


All identified technical difficulties, with an assessment 
from Low to Moderate or High risk, must be accompanied 
by a brief description of the problem, from which the 
needs and magnitude of possible impacts, in terms of 


www.ijaers.com 


International Journal of Advanced Engineering Research and Science, 9(5)-2022 


schedule and costs, may be assessed. Once a given 
architecture is chosen, these identified risks shall be 
treated and mitigated as part of the risk management of the 
technological solution selected. 


3.2.8 STEP 8 


In this step, the technical team shall compute the 
indices listed in Table. 10. The table lists indices whose 
rationale and form of computation are given in Section 3.1. 
All these indices are specific proposals of this article. The 
application of each index for ranking purposes is shown in 
the rightmost column of Table. 10. 


We briefly review the methodology used to compute the 
TDF indices and their applications proposed in the present 
article. From each expert’s assessment form, the 
frequencies of negligible (N), low (L), and moderate or 
high (M/H) assignments are computed. The TDF index for 
each interface is computed from the whole set of forms. 
The system-wide TDF index is then calculated as an 
average of the TDF indices for each interface. A new IRL 
matrix, referred to as the risk-weighted IRL matrix 
(RWIRL), is computed using (7). 
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Table. 9: Austin, M. F., & York, D. M. [19] Decision Criteria for Assessing Integration Readiness Level (IRL) 









































IRL Definition Evidence Description 
: No integration between specified components has been planned or 
0 | No Integration . 
intended. 
; ; š Principal integration technologies have been identified. Top-level 
A high-level concept for integration has : P g g . : P i 
1 been identified functional architecture and interface points have been defined. A high- 
een identified. . ; 
level concept of operations and main use case has been started. 
i rom Inputs/outputs for principal integration technologies/mediums are 
There is some level of specificity of P P ; P P £ a g 
: : known, characterized, and documented. Main interface requirements 
2 | requirements to characterize the erie : . . 
Ean ee bet ‘ and specifications for integration technologies have been 
interaction between components. : 
P defined/drafted. 
The detailed integration design has The detailed interface design has been documented. System interface 
3 | been defined to include all interface diagrams have been completed. Inventory of external interfaces is 
details. completed, and data engineering units are identified and recorded. 
Validation of interrelated functions Integrating technologies (modules/functions/assemblies) has been 
4 | between integrating components in a successfully demonstrated in a laboratory/synthetic environment. 
laboratory environment. Data transport method(s) and specifications have been defined. 
Validation of interrelated functions Individual modules are tested to verify that the module components 
5 | between integrating components in a (functions) work together. External interfaces are well defined (e.g., 
suitable environment. source, data formats, structure, content, method of support, etc.). 
Validation of interrelated functions ; , í 
: . : The end-to-end Functionality of Systems Integration has been 
6 | between integrating components in an j ee 
: ; validated. Data transmission tests were completed successfully. 
appropriate end-to-end environment. 
A fully integrated prototype has been successfully demonstrated in 
System prototype integration an actual or simulated operational environment. Each 
7 | demonstration in an operational high- system/software interface is tested individually under stressed and 
fidelity environment. abnormal conditions. Interface, Data, and Functional Verification 
complete. 
System integration completed and . ee , , 
y . g p A fully integrated system can meet overall mission requirements in 
mission qualified through test and . . . a 
8 Se : an operational environment. System interfaces qualified and 
demonstration in an operational is . . : 
f functioning correctly in an operating environment. 
environment. 
ON A fully integrated system has demonstrated operational 
System Integration is proven through : ia AN : 
i s effectiveness and suitability in its intended or a representative 
9 | successful mission-proven operations , , f 
biliti operating environment. Integration performance has been fully 
capabilities. . : : è ; 
p characterized and is consistent with user requirements. 











The technical team shall choose the values of the 
weights Wy, W and Wmm. A few prescriptions apply. For 
consistency, the value of the Technical Difficulty Factor 
index for an interface shall increase with increasing 
integration risk. Hence, the values given to the weights 


shall obey the relations: 
Wy < W, < Wun. (14) 


A natural choice for W,is (Wy + Wyju). This choice 


considers a linear relation between option and weight. 
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Since the prescription for deriving system indices from 
system element indices is still an open issue [24, 22], 
below, we give alternative ways of computing the index 
RWSRL, following different prescriptions given in the 
literature. 
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Table. 10: Ranking indices and their use according to the 
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proposed framework 
Imq(2)_ [Severity of risk |Ranking of the risks m for 
m for the each interface g, according 
integration of _|to risk severity. 
interface q. 
TDF, Technical Ranking of interfaces g for 
(5) Difficult Factor |a given system architecture, 
associated with Jaccording to assessed 
interface q. interface risk of integration. 
TDF System-wide |Ranking of architectures, 


(6) Technical 


according to interface risk 





Difficult of integration. 
Factor. 
RWIRL,  |Risk-weighted |Ranking of the interfaces q 
(7) interface for a given system 


readiness level 
for interface q. 


architecture, according to 
composite measure of 
Interface Readiness Level 
and assessed interface risk 
of integration. 





RWSRL,  |Risk-weighted |Ranking of the elements i 
(9) readiness level jof a system architecture 
for component |according to a composite 
i. measure of the attributes 
equipment i maturity, 
integration maturity of the 
equipment i interfaces and 
severity of perceived 
integration risks associated 
with equipment i interfaces. 
RWSRL |System-wide |Ranking of architectures, 


according to a composite 
measure of the attributes of 
readiness level. |maturity of system’s 
equipment, maturity of 
system’s interfaces, and 
risk associated with the 
integration of system’s 
interfaces. 


(12) risk-weighted 
interface 

















RWIRL_int |System-wide 
(8) risk-weighted 

interface 

readiness level. 


Ranking of architectures, 
according to a composite 
measure of the attributes: 
(a) integration maturity of 
the equipment’s interfaces 
and (b) severity of 
perceived integration risks 
of equipment’s interfaces. 











RWIRL,  |Risk-weighted |Ranking of equipment 
(11) interface according to its interface 
readiness level |maturity and the perceived 
for equipment |difficulty of its integration 
i. to the system. 
RWIRL [System-wide [Ranking of architectures 


(13) risk-weighted 
equipment 
readiness level 
for integration 
to the system. 








according to equipment 
interface readiness for 
integration and perceived 
equipment integration risk. 
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3.2.8.1 Prescription by Ross 
Here, the computation of the RWSRL is adapted from the 
SRL prescription given by Ross[27]. 


For a system with n elements, 


prescription is as follows: 


the complete 


A -assign a TRL index to each system element, 
according to the scale given in Table 8, and express the 
system TRL in the form of a vector: 


TRL = (trh, trl, apt ; (15) 


B —for the considered integration type (M, E, T or D/C) 
assign an IRL index to each identified interface in the 
system, according to the scale given in Table. 9, and 
express the result in matrix form as: 


itla irlo e thn 
IRI* = irl21” irl,* mia irlan” , (16) 
irl” itlyg” irlan” 


where irli* identifies the index corresponding to the system 
elements i and j, for the integration of type x; the diagonal 
elements, which give the index corresponding to the 
integration of a component with itself, are not used in this 
SRL computation prescription; 


C -from the elements of the matrix]RL*, compute the 


matrix below, the rationale of which is given in Section 
3.1: 


(Wyt+ Wm/H- TDFij) 


rwirl*® = irl} x 
J J (Wm/H-WN) 


> (17) 


where TDF;; is given by (5) and Wy and Wy ;yare weights 


used in the interface risk analysis, defined in the Step 8; 
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D — compute the average value of the matrix elements in 
(17) as: 


. x 1 * x 

rwirl,” = ~œ ) rwirli;”, (18) 
+ ji 

where Q is the number of non-zero elements in the 

considered line; 


E — assign a manufacturing readiness level (MRL) 
index to each system element, according to a proper 
scale[27], and express the system MRL in the form of a 
vector: 


MRL = =(mrl,mrly wig iL, J; (19) 


F — compute the component system readiness level 
vector: 


RWSRL* = 20 
(rwsrl,”, rwsrlz” ...,TwsTln”) , (20) 
from the expression: 
1 
rwsrl;* = mrl; x trl; X gwir” ; (21) 
G — finally, compute the system readiness 


levelIRWSRL*”for the integration of type x, as the average 
of the elements of the vector given in (18): 


RWSRI* = <rwsrl;*. (22) 


It should be noted that the index rwsrl;*is equivalent 
to the index RWSRL,defined in (9), with the difference 
that here the type of integration x is being considered in an 
explicit way. When the system's topology does not change 
with the integration type (M, E, T or D/C) such explicit 
distinction is not necessary, being then possible to refer to 
each interface by a single index q, allowing for the 
multiplicity introduced by the different integration types. 


The sequence of calculations leading to RWSRL*is 
carried out for the four types of integration. The RWSRL 
index of the system is finally computed from the equation: 


RWSRL™ + RWSRLE + RWSRL™ + RWSRL?/¢ 
RWSRL = ji (23) 





The above computation should be carried out for all 
architecture solutions considered in the analysis. 


It should be noted that (23) is equivalent to (12) if one 
considers the manufacturing readiness level for each 
component equal to its maximum value, i.e., if there are no 
limitations as regards manufacturing of system elements. 
Thus, the methodology given in Section 3.1, based on the 
theory of ranking a variable with multiple attributes [36, 
37], is equivalent to the approach of Ross[27] for 
computing a system readiness level from the TRL and IRL 
of system elements. 
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3.2.8.2 Prescription by Sauser 


The presentation given here is based on the work of 
Austin, M.F., & York, D. M. [19] and follows the 
prescription given by Sauser [8]. 


The complete prescription is as follows: 


A — for each integration type x (M, E, T e D/C), assign 
an IRL index to each identified interface in the system, 
according to the scale given in Table. 9, and compute the 
rwirl;, matrix from (17); in the present case, each 


diagonal element of IRL is assigned a value of 9; 


B -the RWSRL vector is obtained by multiplying the 
normalized TRL vector by the normalized RWIRL matrix: 


n 


rwirli trl 
rwsrl¥ =” 9 š” (24) 





j=1 
where the normalization factor is taken as the maximum of 
the corresponding assignment scales; 


C —compute the component RWSRL vector from the 
expression: 


crwsrl* =rwsrl* / mi, (25) 
where mj; is the number of integrations of component i, as 


defined by the system architecture, including the 
integration of the component with itself; 


D — the arithmetic average of the elements of the 
component RWSRL gives the composite RWSRL, which is 
interpreted as a measure of the system readiness, is then 
provided by: 

1 n 
RWSRL* = -y crwsrlj, (26) 
i=1 
where n is the number of elements in the considered 
architecture. 


The sequence of calculations must be repeated for the 
four types of integration, and the RWSRL of the system is, 
finally, computed from the equation (23): 


IV. DISCUSSION 


Multi-attribute Decision-making (MADM) is 
conceptually considered a branch of the area of Multi- 
criteria Decision-making (MCDM) in the field of 
Operations Research[39]. 


We note that the general form of a weighted 
multiplicative scoring function, with k attributes, is given 


by: 
F= | [a ' (24) 
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5 y= 1, (25) 


i=1 
where w designates the weight exponent of the index 
associated with the attribute a;. If all y are equal, they 
may be dropped from (24) and (25). All the weighted 
multiplicative scoring functions defined in Section 3.1 use 
equal weight exponents. In principle, the formalism may 
be further developed through a judicious choice of the 
weight exponents, considering the specificities of the 
intended application. It should also be noted that the 
effective scoring functions leading to the indices RWIRL 
and RWSRL are composed of a mix of multiplicative and 
additive scoring functions. These indices should be 
carefully analyzed for each specific application, given the 
possible idiosyncrasies affecting additive scoring 
functions, as discussed in detail by Tofallis [38]. 


The framework given in this article has not explicitly 
considered an index for the attribute of technology 
maturation risks. Such risks might be assessed through an 
AD2 framework, as already discussed in Section 3.1. We 
emphasize that the AD2 methodology investigates the risk 
areas of design and analysis, manufacturing, software 
development, testing, and operations, which may be 
considered complementary to the risk area considered in 
the present framework. In principle, an index associated 
with the attributes associated with technology maturation 
risk might be devised and implemented following the 
concepts given in this article. 


Fig. 7 Error! Reference source not found.shows the 
relation between the AD2 level scale and risk severity, 
represented by its two components: impact and probability. 
The chart in the lower part of the figure gives descriptions 
for the classification of risks’ impact into five levels, while 
the upper chart gives the correspondence of each AD2 
scale level with corresponding values of impact level and 
probability (severity). 


The framework proposed in this article may be 
interpreted as a version of the AD2 framework, adapted for 
the assessment of integration risks. To further ascertain 
this point, Fig. 8 gives the relation between the assessment 
of the selected integration risks listed in Table. 2 and risk 
severity, represented by the pair impact and probability. 
Comparing with the charts given in Fig. 7, it is seen that 
the present framework and the AD2 framework are 
conceptually equivalent, although applied to the different 
risk areas. 
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Requires new development outside of any existing 
experience base. No viable approaches exist that can be 
pursued with any degree of confidence. Basic research} 90% 
in key areas needed before feasible approaches can be 
defined. 


Requires new development where similarity to existing 
experience base can be defined only in the broadest]/80% 
sense. Multiple development routes must be pursued. 


Requires new development but similarity to existing 
experience is sufficient to warrant comparison in only a 
subset of critical areas. Multiple development routes must 

be pursued. 

Requires new development but similarity to existing 
experience is sufficient to warrant comparison on only a 
subset of critical areas. Dual development approaches 
should be pursued in order to achieve a moderate degree] 50% 
of confidence for success. (desired performance can be 
achieved in subsequent block upgrades with high degree} 

of confidence. 

Requires new development but similarity to existing 
experience is sufficient to warrant comparison in all 
critical areas. Dual development approaches should be] 40% 
pursued to provide a high degree of confidence for 
SUCCESS. 

Requires new development but similarity to existing 
experience is sufficient to warrant comparison across the| 
board. A single development approach can be taken with 

a high degree of confidence for success. 


base. A single development approach is adequate. 
development approach is adequate. 


Exists with no or only minor modifications being required. 
equate. 


[Seteaue | Schedule 


np 3 Minimal or no 
M I 
inimal or no impact jiga 


Minor reduction in Budget 
technical increase or unit 


Minimal or no 
consequence to 


Able to meet key 
dates. 
Slip <*month(s) 


performance or 
supportability, can be increases. 
tolerated with little or no <**(1% of 
impact on the program Budget) 
Minor schedule slip. 
Able to meet Budget 
key milestones with | increase or unit 
no schedule float | production cost 
Slip <*month(s) increases. 
Sub-system <**(5% of 
slip<*month(s) plus Budget) 
available float 
Significant degradation in Budget 
technical performance or = increase or unit 
major shortfall in ee production cost 
supportability; may Slip <*month(s) increases. 
jeopardize program <**(10% of 
SUCCESS Budget) 
Severe degradation in Budget 
technical performance. increase or unit 
Cannot meet KPP or key production cost 
technicalVsupportability increases. 
threshold; will jeopardize >**(10% of 
program success Budget) 
DOD Descriptions 


production cost 


Moderate reduction in 
technical performance or 
supportability with limited 

impact on program 
objectives 


Cannot meet key 
program milestones 
Slip <*month(s) 


QI 














Fig. 7: Relation between the AD2 framework and the risk 
management process in a project. Adapted from Bilbro[7] 
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Extremal 5 - Near certainty: ~ 90 % 
High 4 - Highly likely: ~ 70 % 
Medium 3 - Likely: ~“ 50 % 

Low 2 - Low likelihood: ~ 30 % 
Minimal 1- Not likely: ~“ 10% 





Fig. 8: Relation between the proposed framework and the 
risk management process in a project. Colors indicate the 
severity level, according to the scale depicted in Fig. 3 


Using the concepts introduced in this article makes it 
possible to associate an index to each of the risks 
investigated in the AD2 framework. As an illustration, Fig. 
9 gives a possible correspondence betweenAD2 scale 
levels with the computation concepts introduced in this 
article. 
































AD2 levels Risk Weights 
Options 

9 

8 M/H Wmm 
7 

6 

5 L WL 
4 

3 

2 N Wn 
1 

















Fig. 9: Possible relation between the AD2 scale levels and 
the framework presented in this article. 


Legend: N: negligible; L: low; M/H: moderate to high 


In order to accommodate the AD2 scheme into the 
present framework, it would then be necessary to 
formulate the AD2 specific questions as risk statements 
and unfold them into three excluding possibilities, 
following the logic illustrated in Table. 2. Instead of a 
form for each interface, there would be a form for each 
element of the system work breakdown structure (WBS). 
From this point on, a TDF factor would then be computed 
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for each WBS element and composed with the other 
attributes, as discussed in Section 3.1, giving origin to 
different indices. This additional step would complete the 
procedure of the framework proposed in this article. The 
outcome is a project technology assessment framework 
that composes the following technology readiness 
assessments and risks: technology readiness level of 
equipment, technology readiness level of interfaces, 
technology maturation risks, and integration risks. 


V. RESULTS 


This section illustrates the application of the proposed 
framework. Two examples are developed in order to show 
that the proposed methodology can capture relevant 
information from the system through indices, which are 
consistent with the information from the analysis. 


5.1 Example 1 


An example involving one interface type, or a 
configuration in which the architectures for the different 
interface types are topologically equivalent, is presented in 
the following. Table. 11 shows the parameter values used 
in the example. 


Table. 11: Parameters’ values for Example 1 


32 (Co. |< —+{ oa 
Al 
~A 7 as 
4 \ X 
E N 
X 
i b i 
( E(5) AA. c3) ) 
T J \ J 
a \ rr" 
w 
> 


© 








Number of 
Evaluators (Na) 





Number of system 
components (n) 





Number of 
interfaces (Q) 























Wn 0.8 
WL 1.0 
Wmm 1.2 
TRL 3 9 9 4 7 
IRL matrix 0 7 9 3 7 
0 0 0 1 
0 3 2 
0 0 
0 























The assessment of the 7 interfaces (Q) by 32evaluators 
(Na) has been simulated by randomly selecting an 
assignment for each risk. Fig. 10 gives the aggregated 
result for each risk, for Interface 1, as an example. Similar 
results are obtained for the other interfaces. 
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Identification: INTERFACE 1 


















1 - Know-how 


0,9777 














2-GSE 0,9893 
Zza OO 0,9955 
“4-Workload 0,9911 
“5-Trainng 0,9893 





1,0179 
1,0000 













9 - Complexity 








Frequency 





Weighting factors 











Criterion's weight 


TDF 











Fig. 10: Aggerated result for Interface 1, according to the 
simulated assessment of Na = 32 evaluators. The cells in 
red and in green identify what would be the most and least 
severe risks, as assessed by the technical personnel 


The rightmost column shows the values of the 
indexl,,,for each risk. The values give a measure, for 
ranking purposes only, of the relevance of the 
corresponding risk (m) for the considered interface 
integration (q). The highlighted maximum (red) and 
minimum (green) values correspond to the most and least 
severe risks for the assessed interface. 







































































Interface IRL TDF, RWIRL, 
1 7 0,9972 7,0194 
2 9 0,9944 9,0500 
3 3 1,0021 2,9938 
4 7 0,9986 7,0097 
5 1 1,0063 
6 3 0,9896 3,0313 
7 2 1,9847 
TDF 0,9994 
RWIRL int 45832 











Fig. 11: The indices TDFq and RWIRLg are given for each 
interface for the simulated example discussed in the text. 
The system-wide indices TDF and RWIRL_int are also 
shown 

The JRL, TDF, and RWIRL, indices for each interface 
are given in Fig. 11, with the least and most favorable 
cases identified with red and green colors. The figure also 
lists the corresponding system-wide indices, TDF and 


www.ijaers.com 


International Journal of Advanced Engineering Research and Science, 9(5)-2022 


RWIRL_int, for the simulated configuration. When 
considering only the difficulties perceived by the technical 
team, i.e., when considering only the index TDF, this 
simulated example would show that interface 7, with IRL 
equal to 2, would rank first in risk severity when compared 
to the other interfaces. Interface 6, although with a low 
IRL value, equal to 3, would present the most favorable 
situation within the considered set of risks. When 
composing integration risk with maturity, interface 5, with 
the lowest IRL, equal to 1, would be the most challenging 
interface, while interface 2, with IRL equal to 9, would 
represent the most favorable case. 





























Equipment/ | TRL | RWSRL; RWIRL,; 
Interfaces 

1 / 1-2-3-4 3 19,5547 6,5182 
2/1-5 9 36,0594 4,0066 
3 / 2-6-7 9 42,1979 4,6887 
413-6 4 

5 / 4-5-7 7 23,3058 3,3294 

RWSRL 26,63355 

















Fig. 12: The index RWSRLi for each equipment is given. 
The last line shows the system-wide index RWSRL 


Fig. 12 gives the values of RWSRL; andthe average 
values of the risk-weighted integration level, RWIRL;, 
associated with each element i. The figure also gives the 
value of the system-wide index RWSRL. In the simulated 
scenario, with random assessments of each interface, the 
system component ranked as most critical has TRL 4 and 
coincides with the element that shows the most 
unfavorable value for the index RWIRL,;. 


5.2 Example 2 


The hypothetical case of a system with five component 
equipment with four alternatives for one of the 
components is now studied through simulation. 


It is assumed that the system displays different 
configurations for different interface types. Fig. 13 shows 
the assumed configurations for the four types of interfaces 
defined in Table. 4 (M, E, T, and D/C). Ordinal numbers 
identify interfaces, while system components are identified 
by letters, sequentially from A to E. Alternative candidate 
equipment, for the “A” position, are identified by Al 
through A4. 
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Fig. 13: System decomposed into the four types of 
interfaces. 


TRL and IRL values for equipment and interfaces have 
been randomly selected and are displayed in Fig. 14, with 
the provision that the IRL indices are the same for each 
type of interface (M, E, T, and D/C), even when switching 
between candidates for the “A” position. Also, it is 
assumed that the insertion of candidate equipment does not 
affect the TRL of the remaining system equipment B, C, 
D, and E. The assessment of the 9 risks by 3 specialists has 
also been randomly selected. Although the specific 
simulated situation would seldom be verified in actual 
cases, the conceptual case of choosing a piece of 
equipment among several possibilities is quite common. 
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Fig. 14: Simulation input data. 
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Fig. 15 lists theRWSRL index for each type of 
interface, referred to as RWSRL(x), with x e (M,E,T,D/ 
C), as well as the systemRWSRL for each architectural 
solution formed with the candidate components. It is seen 
that the solution with the equipment A4 presents the 
highest RWSRL value. Comparing the RWSRL(x) values 
among the different solutions, it is observed that Solution 4 
ranks first, except for the D/C interface type, when 
Solution 3 ranks first. Concerning the TDF index, which 
would give a measure of integration risk for each 
configuration as perceived by the technical team, Solution 
4 also ranks first as the preferred solution (lower risk). It is 
also seen from Fig. 14 that the alternatives A3 and A4 
show TRL values equal to 7, superior to the value 6 for 
alternatives Al and A2. From Fig. 14, it is also seen that 
the average IRL for component A4, equal to 22/3, is 
superior to the equivalent values for alternatives Al to A3, 
equal to 22/4, 9/2, and 9/2, respectively. Thus, from the 
analysis of the basic input indices it would be expected 
that alternative 4 would fare better than the other 
alternatives. The virtue of methodologies belonging to the 
MADM class is to condense, through one or more indices, 
computed straightforwardly, information that would be 
obtained through detailed analysis. In this example, the 
proposed methodology would give, through the language 
of ranking indices, the same answer as would be obtained 
by a detailed analysis of the problem. 







































































Composition RWSRLsys | TDF sys 
A1-B-C-D-E 
RWSRL (M) 28,3062 
Architecture Solution 1 RWSRL (E) 25,6551 
RWSRL (T) 35,6578 33,0733 | 1,0123 
RWSRL (D/C) 42,6741 
Composition 
RWSRLsys | TDF sys 
A2-B-C-D-E 
Architecture Solution 2 RWSRL (M) 27,9211 
RWSRE (E 26,5911 33,5081 | 0,9990 
RWSRL (T) 36,2514 
RWSRL (D/C) 43,2689 
ae ae RWSRL sys | TDF sys 
Architecture Solution 3 RWSRL (M) 28,5743 
See) 27 8088 34,6057 | 1,0045 
RWSRL (T) 36,0402 á i 
RWSRL (D/C) 46,3795 
eu RWSRL sys | TDF sys 
A4-B-C-D-E 
Architecture Solution 4 RW SEE) EW 
EWSRL (E) ze 34,9820 | 0,9890 
RWSRL (T) 36,4193 4 i 
RWSRL (D/C) 45,7548 


Fig. 15: Result of the ranking simulation of the 
architectural solutions formed between candidates Al, 
A2, A3 and A4. 
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The TDF index also synthesizes relevant information 
for the project risk management process, pointing out 
which risks require more attention from project 
management. 
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Fig. 16: Ranking result of the architecture solution 4 
by risk type and by interface type 


Fig. 16 shows a comparison of the relevance of each 
assessed risk for each interface type as given by the Ing 
index, for Solution 4. For instance, risk 9, “Complexity”, is 
ranked first for the Signal interface, while risk 
“Verification” ranks first for the Thermal interface. 
Among all risks, the risk GSE ranks first, for the case of an 
Electrical interface type, according to the technical team 
risk assessment leading to the TDF index. 


When the objective is to understand which type of 
interface presents the highest integration risk, the 
Technical Difficulty Factor (TDF) may be computed for 
each interface type. This index makes it possible to rank 
the different interface types according to their integration 
risk. For the example under scrutiny, Fig. 17shows that the 
simulated data would indicate that the thermal and signal 
interface types would require greater attention than the 
other interface types in the integration effort. 
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Fig. 17: Ranking of TDF by interface types of architecture 
solution. 


VI. CONCLUSION 


The methodology given in this article aims to propose 
ranking indices that measure the integration risks of 
system architectures, to be compounded with system 
technology and interface readiness levels indices to define 
multiple attribute indices adequate for the ranking of 
different system architectures. This objective has been 
implemented as follows. 


The integration risk assessment is based on specialized 
opinion. In the methodology version presented here, a 
selection of nine integration risks, covering a broad range 
of disciplines, is submitted, in the format of an individual 
form survey, to the assessment of systems engineering 
personnel, specialty engineers, project managers, and 
assembly, integration and testing specialists. The detailed 
risk information that emerges from the survey is translated 
into an index, referred to as the Technical Difficulty Factor 
(TDF), which gives a measure of integration risk severity 
for each interface. 


Through the composition of this index with the set of 
IRL indices, a new composite index, referred to as Risk- 
weighted Integration Readiness Level (RWIRL), is defined 
for each interface. The RWIRL indices are then 
compounded with the equipment TRL indices, giving 
origin to a new set of indices, referred to as RWSRLi, 
which may be used to rank system equipment according to 
the attributes of equipment maturity, interface maturity, 
and integration risk. From the set of RWSRLi indices, a 
system-wide index, termed Risk-weighted System 
Readiness Level (RWSRL), is derived, which may be used 
to rank different system architectures, considering the 
attributes mentioned above. 
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With additional effort, through a procedure indicated in 
the article, the TDF index may be compounded with a 
conveniently defined AD2 index, thus defining a 
compounded TDF risk index, which now considers 
integration risks and technology maturation risks. 
Following the procedure already delineated above, one 
obtains equivalent indices as those above defined, 
incorporating both maturity and risk information. 


The outcome is a project technology assessment 
framework that composes the following elements: 
equipment readiness level, technology readiness level of 
interfaces, technology maturation risks, and integration 
risks. 


It is pointed out that the RWSRL methodology 
proposed in this article deals with the definition of ranking 
indices and their applications. Values of computed indices 
have meaning only for comparison purposes; it is not 
possible to assign them a scale as in the TRL, IRL, and 
SRL methodologies. The variability of the results 
generated will always depend on the values adopted for the 
weights Wn, WL, and Wmm. The values may be tailored 
according to the conveniences of the intended application. 
This will not affect the result whenever linearity between 
them is observed. 


Although conceived for application in space projects” 
Phases A and B, the RWSRL methodology may be 
applicable, in principle, to a wide range of project types, at 
different life cycle stages, for instance, whenever a 
decision among multiple supply options is necessary or as 
an input for related RIDM processes. The proposed risk 
assessment model is sufficiently generic to be tailored to 
other design applications. 


The various ranking indices proposed in the article 
provide potentially relevant data for project management, 
not only as a selection tool among candidate items for the 
system's composition but also as an input for a 
conventional CRM process for detailing and mitigating 
risks. The proposed risk assessment may indicate the areas 
and aspects related to system integration that require great 
or special attention in the system design. 


There is a relationship between the AD2 scale levels 
and the structure presented in this article, as shown in Fig. 
8 and Fig. 9. It can be argued that both tools work 
similarly and that the AD2 and RWSRL methodologies 
may work as complementary frameworks. 


The RWSRL metric captures the potential risks at a 
given moment of the project, as perceived by the technical 
team. Beyond their utility in decision-making processes, 
the proposed indices provide an effective communication 
bridge between systems teams and project management. 


They convey appropriate identification and 
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characterization of integration difficulties and risks 
foreseen in each evaluated architecture solution. 


The utility of miulti-attribute decision-making 
methodologies as applied to projects is to condense, 
through one or more indices, computed straightforwardly 
and systematically, information that would be obtained 
through detailed analysis. The examples given in Section 3 
indicate that the proposed methodology is successful in 
providing through the language of ranking indices 
information that would be obtained only through a detailed 
analysis. 
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